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1 Introduction 

Wavelets are a mathematical tool for hierarchically decomposing 
functions. They allow a function to be described in terms of a coarse 
overall shape, plus details that range from broad to narrow. Regard- 
less of whether the function of interest is an image, a curve, or a sur- 
face, wavelets offer an elegant technique for representing the levels 
of detail present. This primer is intended to provide people working 
in computer graphics with some intuition for what wavelets are, as 
well as to present the mathematical foundations necessary for study- 
ing and using them. In Part 1, we discuss the simple case of Haar 
wavelets in one and two dimensions, and show how they can be used 
for image compression. In Part 2, we will present the mathematical 
theory of multiresolution analysis, then develop spline wavelets and 
describe their use in multiresolution curve and surface editing. 

Although wavelets have their roots in approximation theory [5] and 
signal processing [13], they have recently been applied to many 
problems in computer graphics. These graphics applications in- 
clude image editing [1], image compression [6], and image query- 
ing [10]; automatic level-of-detail control for editing and render- 
ing curves and surfaces [7, 8, 12]; surface reconstruction from con- 
tours [14]; and fast methods for solving simulation problems in ani- 
mation [11] and global illumination [3, 4, 9, 15]. For a discussion of 
wavelets that goes beyond the scope of this primer, we refer readers 
to our forthcoming monograph [16]. 

We set the stage here by first presenting the simplest form of 
wavelets, the Haar basis. We cover one-dimensional wavelet trans- 
forms and basis functions, and show how these tools can be used to 
compress the representation of a piecewise-constant function. Then 
we discuss two-dimensional generalizations of the Haar basis, and 
demonstrate how to apply these wavelets to image compression. 

Because linear algebra is central to the mathematics of wavelets, we 
briefly review important concepts in Appendix A. 

2 Wavelets in one dimension 

The Haar basis is the simplest wavelet basis. We will first discuss 
how a one-dimensional function can be decomposed using Haar 
wavelets, and then describe the actual basis functions. Finally, we 
show how using the Haar wavelet decomposition leads to a straight- 
forward technique for compressing a one-dimensional function. 

2.1 One-dimensional Haar wavelet transform 

To get a sense for how wavelets work, let's start with a simple exam- 
ple. Suppose we are given a one-dimensional "image" with a reso- 
lution of four pixels, having values 

[ 9 7 3 5 ] 
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We can represent this image in the Haar basis by computing a 
wavelet transform. To do this, we first average the pixels together, 
pairwise, to get the new lower resolution image with pixel values 

[8 4] 

Clearly, some information has been lost in this averaging process. 
To recover the original four pixel values from the two averaged val- 
ues, we need to store some detail coefficients, which capture the 
missing information. In our example, we will choose 1 for the first 
detail coefficient, since the average we computed is 1 less than 9 
and 1 more than 7. This single number allows us to recover the first 
two pixels of our original four-pixel image. Similarly, the second 
detail coefficient is — 1, since 4 + (— 1) = 3 and 4 — (— 1) = 5. 

Thus, we have decomposed the original image into a lower resolu- 
tion (two-pixel) version and a pair of detail coefficients. Repeating 
this process recursively on the averages gives the full decomposi- 
tion: 

Resolution Averages Detail coefficients 

4 [ 9 7 3 5 ] 

2 [84] [ 1 -1 ] 

1 [«] [2] 

Finally, we will define the wavelet transform (also called the wavelet 
decomposition) of the original four-pixel image to be the single co- 
efficient representing the overall average of the original image, fol- 
lowed by the detail coefficients in order of increasing resolution. 
Thus, for the one-dimensional Haar basis, the wavelet transform of 
our original four-pixel image is given by 

[6 2 1 -1 ] 

The way we computed the wavelet transform, by recursively aver- 
aging and differencing coefficients, is called afilter bank — a process 
we will generalize to other types of wavelets in Part 2 of our tuto- 
rial. Note that no information has been gained or lost by this process. 
The original image had four coefficients, and so does the transform. 
Also note that, given the transform, we can reconstruct the image to 
any resolution by recursively adding and subtracting the detail co- 
efficients from the lower resolution versions. 

Storing the image's wavelet transform, rather than the image itself, 
has a number of advantages. One advantage of the wavelet trans- 
form is that often a large number of the detail coefficients turn out 
to be very small in magnitude, as in the example of Figure 1 . Trun- 
cating, or removing, these small coefficients from the representa- 
tion introduces only small errors in the reconstructed image, giving 
a form of "lossy" image compression. We will discuss this particu- 
lar application of wavelets in Section 2.3, after we present the one- 
dimensional Haar basis functions. 
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Figure 1 A sequence of decreasing-resolution approximations to a 
function (left), along with the detail coefficients required to recapture 
the finest approximation (right). Note that in regions where the true 
function is close to being flat, a piecewise-constant approximation 
works well, so the corresponding detail coefficients are relatively 
small. 

2.2 One-dimensional Haar wavelet basis functions 

We have shown how one-dimensional images can be treated as se- 
quences of coefficients. Alternatively, we can think of images as 
piecewise-constant functions on the half-open interval [0, 1). To do 
so, we will use the concept of a vector space from linear algebra. 
A one-pixel image is just a function that is constant over the entire 
interval [0, 1). We'll let V° be the vector space of all these func- 
tions. A two-pixel image has two constant pieces over the inter- 
vals [0, 1/2) and [1/2, 1). We'll call the space containing all these 
functions V 1 . If we continue in this manner, the space V' will in- 
clude all piecewise-constant functions defined on the interval [0, 1) 
with constant pieces over each of 1 equal subintervals. 

We can now think of every one-dimensional image with 2 pixels as 
an element, or vector, in V' . Note that because these vectors are all 
functions defined on the unit interval, every vector in V' is also con- 
tained in V j+[ . For example, we can always describe a piecewise- 
constant function with two intervals as a piecewise-constant func- 
tion with four intervals, with each interval in the first function cor- 
responding to a pair of intervals in the second. Thus, the spaces V j 
are nested; that is, 

V° C V 1 C V 2 C ••• 



The mathematical theory of multiresolution analysis requires this 
nested set of spaces V . We will consider this topic more thoroughly 
in Part 2. 

Now we need to define a basis for each vector space V 1 . The basis 
functions for the spaces V J are called scaling functions, and are usu- 
ally denoted by the symbol (p. A simple basis for V J is given by the 
set of scaled and translated "box" functions: 

4(x) := (f>(?x -i), i = 0, . . . , 2' - 1, 



where 



<x) := | J 



for 0 < x < 1 
otherwise. 



As an example, Figure 2 shows the four box functions forming a ba- 
sis for V 2 . 

The next step is to choose an inner product defined on the vector 
spaces V . The "standard" inner product, 



■Jo 



(f\g) ■■= / f(x)g(x)dx, 



for two elements f,g € V will do quite well for our running ex- 
ample. We can now define a new vector space W' as the orthogonal 
complement of V' in V i+[ . In other words, we will let W be the space 
of all functions in V i+l that are orthogonal to all functions in V* under 
the chosen inner product. Informally, we can think of the wavelets 
in W as a means for representing the parts of a function in V i+1 that 
cannot be represented in V j . 

A collection of linearly independent functions ^(x) spanning W are 
called wavelets. These basis functions have two important proper- 
ties: 

1 . The basis functions ^ of W, together with the basis functions <^ 
of V, form a basis for V j+i . 

2. Every basis function ^ of W is orthogonal to every basis func- 
tion <jJ t of V' under the chosen inner product. 1 

Thus, the "detail coefficients" of Section 2. 1 are really coefficients 
of the wavelet basis functions. 

The wavelets corresponding to the box basis are known as thsHaar 
wavelets, given by 



where 



^(x) := ip(2lx - i), i = 0, . 



r 1 for 0 < x < 1/2 
iP( x ) ;= ) -1 for 1/2 < x < 1 
! 0 otherwise. 



Figure 3 shows the two Haar wavelets spanning W [ . 

Before going on, let's run through our example from Section 2.1 
again, but now applying these more sophisticated ideas. 

We begin by expressing our original image I(x) as a linear combi- 
nation of the box basis functions in V 2 : 

I{x) = cl(f>l{x) + c\<j>\(x) + c\<j)\{x) + cl<j>\(x). 



1 Some authors refer to functions with these properties aspre-wavelets, 
reserving the term "wavelet" for functions^ that are also orthogonal to each 
other. 
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Figure 2 The box basis for V 2 . 
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Figure 3 The Haar wavelets for W [ . 



A more graphical representation is 

X(x) = 9 x I I 

+ 7 x I I 

+ 3 x I I 

+ 5 x I I 

Note that the coefficients c\, . . . , c\ are just the four original pixel 
values [97 3 5]. 

We can rewrite the expression for l(x) in terms of basis functions 
in V [ and W [ , using pairwise averaging and differencing: 

I{x) = clcj>l{x) + c\cj>\{x) + dli>l(x) + d\i>\{x) 

= 8 x I 1 

+ 4 x I I 

+ ! x 

+ -1 x ' 1__| 

These four coefficients should look familiar as well. 

Finally, we'll rewrite X(x) as a sum of basis functions in V°, W°, 
and W 1 : 

I{x) = cg0gW + 4i>l(x) + 4i,l{x) + d\i,\(x) 
= 6 x I I 



+ 2 x J | 

+ ! x 

+ -1 x ' 1__| 

Once again, these four coefficients are the Haar wavelet transform 
of the original image. The four functions shown above constitute 
the Haar basis for V 2 . Instead of using the usual four box functions, 
we can use tp®, ipl, and ip\ to represent the overall average, the 
broad detail, and the two types of finer detail possible in a function 
in V 2 . The Haar basis for V with j > 2 includes these functions as 
well as narrower translates of the wavelet ip(x). 



Orthogonality 

The Haar basis possesses an important property known as orthog- 
onality, which is not always shared by other wavelet bases. An or- 
thogonal basis is one in which all of the basis functions, in this case 
tpQ, ipQ, ipQ, ip{, . . ., are orthogonal to one another. Note that orthogo- 
nality is stronger than the minimum requirement for wavelets thati/^ 
be orthogonal to all scaling functions at the same resolution level/. 

Normalization 

Another property that is sometimes desirable is normalization. A 
basis function u(x) is normalized if (u \ u) = 1. We can normalize 
the Haar basis by replacing our earlier definitions with 

:= 2 i/2 4>(2>x - i) 
:= 2 i/2 ip(2ix - i), 

where the constant factor of 2^ 2 is chosen to satisfy (u \ u) = 1 for 
the standard inner product. With these modified definitions, the new 
normalized coefficients are obtained by multiplying each old coef- 
ficient with superscript j by 2~ ;/2 . Thus, in the example from the 
previous section, the unnormalized coefficients [6 2 1—1] become 
the normalized coefficients 

t 6 2 75 75 ] 

As an alternative to first computing the unnormalized coefficients 
and then normalizing them, we can include normalization in the de- 
composition algorithm. The following two pseudocode procedures 
accomplish this normalized decomposition: 

procedure DecompositionStep(C: array [l..h] of reals) 
for ! <- 1 to h/2 do 

C[i] <- (C[2; - 1] + C[2i])/V2 
C'[h/2 + i] <- (C[2; - 1] - C[2;])/v/2 
end for 
C<- C 
end procedure 

procedure Decomposition (C: array [l..h] of reals) 
C Cj \fh (normalize input coefficients) 
while h > 1 do 

DecompositionStep(C[i. . h]) 
h <- h/2 
end while 
end procedure 

Now we can work with an orthonormal basis, meaning one that is 
both orthogonal and normalized. Using an orthonormal basis turns 
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out to be handy when compressing a function or an image, which 
we describe next. 



2.3 Application I: Compression 

The goal of compression is to express an initial set of data using 
some smaller set of data, either with or without loss of information. 
For instance, suppose we are given a function/^) expressed as a 
weighted sum of basis functions mix), . . . , u m (x): 



fix) 



Ci Ui(%). 



The data set in this case consists of the coefficients a, . . . ,c m . We 
would like to find a function approximating/fx) but requiring fewer 
coefficients, perhaps by using a different basis. That is, given a user- 
specified error tolerance e (for lossless compression, e = 0), we are 
looking for 



fix) = ^ Ci Uj(x) 



16 out of 16 coefficients 



14 out of 16 coefficients 





12 out of 16 coefficients 



10 out of 16 coefficients 



8 out of 16 coefficients 



6 out of 16 coefficients 



^1 



4 out of 16 coefficients 



2 out of 16 coefficients 



such that m < m and \\fix) — fix)\\ < e for some norm. In general, 
you could attempt to construct a set of basis functionsiTi ,u,~ n that 
would provide a good approximation with few coefficients. We will 
focus instead on the simpler problem of finding a good approxima- 
tion in a fixed basis. 

One form of the compression problem is to order the coeffi- 
cients c\,...,c m so that for every m < m, the first m elements of 
the sequence give the best approximation/^) to fix) as measured 
in the L 2 norm. As we show here, the solution to this problem is 
straightforward if the basis is orthonormal, as is the case with the 
normalized Haar basis. 

Let a be a permutation of 1,. . . ,m, and let/(x) be a function that 
uses the coefficients corresponding to the firstm numbers of the per- 
mutation a: 



fix) = y^CV(Q 



1=1 

The square of the L 2 error in this approximation is 

= (fix)-fix)\fix)-fix)) 




The last step follows from the assumption that the basis is orthonor- 
mal, so (ui | Uj) = 5y. We conclude that to minimize this error 
for any given m, the best choice for a is the permutation that sorts 
the coefficients in order of decreasing magnitude; that is, a satis- 
fies |c CT( i)| > • • • > |c CT(m) |. 

Figure 1 demonstrated how a one-dimensional function could be 
transformed into coefficients representing the function's overall av- 
erage and various resolutions of detail. Now we repeat the process, 
this time using normalized Haar basis functions. We can apply L 



Figure 4 Coarse approximations to a function obtained using L 2 
compression: detail coefficients are removed in order of increasing 
magnitude. 

compression to the resulting coefficients simply by removing or ig- 
noring the coefficients with smallest magnitude. By varying the 
amount of compression, we obtain a sequence of approximations to 
the original function, as shown in Figure 4. 

3 Wavelets in two dimensions 

In preparation for image compression, we need to generalize Haar 
wavelets to two dimensions. First, we will consider how to perform 
a wavelet decomposition of the pixel values in a two-dimensional 
image. We then describe the scaling functions and wavelets that 
form a two-dimensional wavelet basis. 

3.1 Two-dimensional Haar wavelet transforms 

There are two ways we can use wavelets to transform the pixel val- 
ues within an image. Each is a generalization to two dimensions of 
the one-dimensional wavelet transform described in Section 2.1. 

To obtain the standard decomposition [2] of an image, we first apply 
the one-dimensional wavelet transform to each row of pixel values. 
This operation gives us an average value along with detail coeffi- 
cients for each row. Next, we treat these transformed rows as if they 
were themselves an image and apply the one-dimensional transform 
to each column. The resulting values are all detail coefficients ex- 
cept for a single overall average coefficient. The algorithm below 
computes the standard decomposition. Figure 5 illustrates each step 
of its operation. 

procedure StandardDecomposition(C: array [1. . h, 1. . w] of reals) 
for row <— 1 to h do 

Decomposition (C[row, 1. . w]) 
end for 

for col <— 1 to w do 

Decomposition (C[l . . h, col]) 
end for 
end procedure 

The second type of two-dimensional wavelet transform, called the 
nonstandard decomposition, alternates between operations on rows 
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Figure 5 Standard decomposition of an image. 

and columns. First, we perform one step of horizontal pairwise aver- 
aging and differencing on the pixel values in each row of the image. 
Next, we apply vertical pairwise averaging and differencing to each 
column of the result. To complete the transformation, we repeat this 
process recursively only on the quadrant containing averages in both 
directions. Figure 6 shows all the steps involved in the nonstandard 
decomposition procedure below. 



procedure NonstandardDecomposition(C: array [1. . h, 1. . h] of reals) 
C <— C/h (normalize input coefficients) 
while ft > 1 do 

for row <— 1 to h do 

DecompositionStep(C[row, 1. . h]) 
end for 

for col <— 1 to h do 

DecompositionStep(C[l . . h,col]) 
end for 
h <- A/2 
end while 
end procedure 



3.2 Two-dimensional Haar basis functions 

The two methods of decomposing a two-dimensional image yield 
coefficients that correspond to two different sets of basis functions. 
The standard decomposition of an image gives coefficients for a ba- 
sis formed by the standard construction [2] of a two-dimensional 
basis. Similarly, the nonstandard decomposition gives coefficients 
for the nonstandard construction of basis functions. 

The standard construction of a two-dimensional wavelet basis con- 
sists of all possible tensor products of one-dimensional basis func- 
tions. For example, when we start with the one-dimensional Haar 
basis for V 2 , we get the two-dimensional basis for V 2 shown in Fig- 
ure 7. Note that if we apply the standard construction to an orthonor- 
mal basis in one dimension, we get an orthonormal basis in two di- 
mensions. 

The nonstandard construction of a two-dimensional basis proceeds 



Figure 6 Nonstandard decomposition of an image, 
by first defining a two-dimensional scaling function, 

4><t>(x,y) := <f)(x) <p(y), 
and three wavelet functions, 



(f)ip(x,y) 
ipip(x,y) 



= (f>(x)ip(y) 
= ip(x)<p(y) 
= ip(x)ip(y). 



We now denote levels of scaling with a superscript/ (as we did in the 
one-dimensional case) and horizontal and vertical translations with 
a pair of subscripts k and I. The nonstandard basis consists of a sin- 
gle coarse scaling function <jxj)Q fi (x,y):=(jxj)(x,y) along with scales 
and translates of the three wavelet functions (j>tp, tp(j>, and tptp: 



4>^ u {x,y) 
ipcjj u {x,y) 
i>^ u {x,y) 



:= Q><t>iP(2>x-k,2!y- 
:= 2>ip<t>(2>x - k,2>y - 
:= 2ipip(2x - k, 2/y - 



The constant 1 normalizes the wavelets to give an orthonormal ba- 
sis. The nonstandard construction results in the basis for V 2 shown 
in Figure 8. 

We have presented both the standard and nonstandard approaches 
to wavelet transforms and basis functions because both have advan- 
tages. The standard decomposition of an image is appealing be- 
cause it simply requires performing one-dimensional transforms on 
all rows and then on all columns. On the other hand, it is slightly 
more efficient to compute the nonstandard decomposition. For an 
m x m image, the standard decomposition requires 4(m 2 — m) as- 
signment operations, while the nonstandard decomposition requires 
only |(m 2 — 1) assignment operations. 

Another consideration is the support of each basis function, mean- 
ing the portion of each function's domain where that function is non- 
zero. All nonstandard Haar basis functions have square supports, 
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Figure 7 Standard construction of a two-dimensional Haar wavelet 
basis for V 2 . In the unnormalized case, functions are +1 where plus 
signs appear, — 1 where minus signs appear, and 0 in gray regions. 

while some standard basis functions have nonsquare supports. De- 
pending upon the application, one of these choices may be prefer- 
able to the other. 



3.3 Application II: Image compression 

We defined compression in Section 2.3 as the representation of a 
function using fewer basis function coefficients than were origi- 
nally given. The method we discussed for one-dimensional func- 
tions applies equally well to images, which we treat as the coeffi- 
cients corresponding to a two-dimensional piecewise-constant ba- 
sis. The approach presented here is only introductory; for a more 
complete treatment of wavelet image compression, see the article 
by DeVore et al. [6]. 

We can summarize wavelet image compression using theL 2 norm 
in three steps: 

1. Compute coefficients ci, . . . , c m representing an image in a nor- 
malized two-dimensional Haar basis. 

2. Sort the coefficients in order of decreasing magnitude to produce 
the sequence c ct( d, . . . , c CT(m) . 

3. Starting with m = m, find the smallest in for which 



Em , 



\ 2 <r J- 



where e is the allowable L error. 



The first step is accomplished by applying either of the two- 
dimensional Haar wavelet transforms described in Section 3.1, be- 
ing sure to use normalized basis functions. Any standard sorting 
technique will work for the second step. However, for large images 
sorting becomes exceedingly slow. 

The pseudocode below outlines a more efficient method that uses 
a binary search strategy to find a threshold below which coefficient 
sizes are deemed negligible. The procedure takes as input a one- 
dimensional array of coefficients C (with each coefficient corre- 
sponding to a two-dimensional basis function) and an error toler- 
ance e. For each guess at a threshold t, the algorithm computes the 
square of the L 2 error that would result from discarding coefficients 
smaller in magnitude than r . This squared error s is compared to e 2 
at each iteration to decide whether the binary search should continue 
in the upper or lower half of the current interval. The algorithm halts 
when the current interval is so narrow that the number of coefficients 




l>o,i(x,y) <frl>\,i(x>y) V>V>d,i<X>') ^\,\{x,y) 
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Figure 8 Nonstandard construction of a two-dimensional Haar 
wavelet basis for V 2 . 



to be discarded no longer changes. 

procedure Compress(C: array [1. . m] of reals; e. real) 
Tmin «- min{ \C[i]\ } 
<- max { | C[i] | } 

do 

T <- (T mi „ + T max )/2 

s <- 0 

for i <— 1 to m do 

if | C[d| < t then s <- s + (C[i]) 2 
end for 

if s < t 2 then T m i„ <— t else T max t 

Until Tmi„ Tmax 

for i <— 1 to m do 

if | C[;] | < t then CM <- 0 
end for 
end procedure 

This binary search algorithm was used to produce the images in 
Figure 9. These images demonstrate the high compression ratios 
wavelets offer, as well as some of the artifacts they introduce. 

DeVore et al. [6] suggest that the L 1 norm is best suited to the 
task of image compression. Here is a pseudocode fragment for a 
"greedy" L 1 compression scheme: 

for each pixel (x, y) do 

S[x,y]<^0 
end for 

for (' <— 1 to m do 

6' <— 5 + error from discarding CM 

if Ex,, \ s 'i x <yi\ < ethen 

discard coefficient CM 
S <- 5' 
end if 
end for 

Note that this algorithm's results depend on the order in which coef- 
ficients are visited. Different images (and degrees of compression) 
may be obtained from varying this order — for example, by start- 
ing with the finest scale coefficients, rather than the smallest coef- 
ficients. You could also run a more sophisticated constrained op- 
timization procedure to select the minimum number of coefficients 
subject to the error bound. 
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(a) (b) (c) (d) 

Figure 9 L? wavelet image compression: The original image (a) can be represented using (b) 19% of its wavelet coefficients, with 5% relativ£ 2 
error; (c) 3% of its coefficients, with 10% relative/, 2 error; and (d) 1% of its coefficients, with 15% relative/, 2 error. 



4 Conclusion 

We have described Haar wavelets in one and two dimensions as well 
as how to use them for compressing functions and images. Part 2 
of this primer will continue this exposition by presenting the math- 
ematical framework of multiresolution analysis. We will also de- 
velop a class of wavelets based on endpoint-interpolating B-splines, 
and describe how to use them for multiresolution curve and surface 
editing. 
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A Linear algebra review 

The mathematics of wavelets rely heavily on fundamental ideas 
from linear algebra. This appendix reviews a few important ideas. 

A.l Vector spaces 

The starting point for linear algebra is the notion of a vector space. 
A vector space (over the reals) can be loosely defined as a collec- 
tion V of elements where 

1. For all a, b € R and for all u, v 6 V, au + bv £ V. 

2. There exists a unique element 0 € V such that 

• for all u € V, Ou = 0, and 

• for all u € V, 0 + u = u. 

3. Other axioms (omitted here) hold true, most of which are neces- 
sary to guarantee that multiplication and addition behave as ex- 
pected. 

The elements of a vector space V are called vectors, and the el- 
ement 0 is called the zero vector. The vectors may be geometric 
vectors, or they may be functions, as is the case when discussing 
wavelets and multiresolution analysis. 

A.2 Bases and dimension 

A collection of vectors u\, m, ■ ■ ■ in a vector space V are said to be 
linearly independent if 



c\U\ + C2U2 + ■ ■ ■ = 0 if and only if c\ = C2 



0. 



A collection « i, «2, • • • € Voflinearly independent vectors is abasis 
for V if every v 6 V can be written as 



v = c 'i u > 



for some real numbers a, c-i, ■ ■ ■ . The vectors in a basis for V are 
said to span V. Intuitively speaking, linear independence means that 
the vectors are not redundant, and a basis consists of a minimal com- 
plete set of vectors. 

If a basis for V has a finite number of elements ui, . . . , u m , then V 
is finite -dimensional and its dimension is m. Otherwise, V is said to 
be infinite-dimensional. 

Example: R 3 is a three-dimensional space, and e\ = 
( 1 , 0, 0), e 2 = (0, 1 , 0), e 3 = (0, 0, 1 ) is a basis for it. 

Example: The set of all functions continuous on [0, 1] is 
an infinite-dimensional vector space. We'll call this space 
C[0, 1]. 



A. 3 Inner products and orthogonality 

When dealing with geometric vectors from the vector space R 3 , the 
"dot product" operation has a number of uses. The generalization of 
the dot product to arbitrary vector spaces is called an inner product. 
Formally, an inner product { • | • ) on a vector space V is any map from 
V x V to R that is 

1. symmetric: {u \ v) = {v \ u), 

2. bilinear: (au + bv \ w) = a(u \ w) + b(v \ w), and 

3. positive definite: (u\u) > 0 for all u ^ 0. 



A vector space together with an inner product is called, not surpris- 
ingly, an inner product space. 

Example: It is straightforward to show that the dot product 
on R 3 defined by 

((ai,fl2,fl3) | Q>i,b2,bi)) := a\b\ + a 2 b 2 + a 3 b 3 (1) 

satisfies the requirements of an inner product. 

Example: The following "standard" inner product on 
C[0, 1] plays a central role in most formulations of multires- 
olution analysis: 

(f\g) ■= I f(x)g(x)dx. 
Jo 

The standard inner product can also be generalized to include 
a positive weight function w(x): 



(f\g) 



■= / Mx)f(x. 
Jo 



) g(x) dx. 



One of the most important uses of the inner product is to formalize 
the idea of orthogonality. Two vectors u, v in an inner product space 
are said to be orthogonal if (u \ v) = 0. It is not difficult to show 
that a collection U\,U2, ■ ■ ■ of mutually orthogonal vectors must be 
linearly independent, suggesting that orthogonality is a strong form 
of linear independence. An orthogonal basis is one consisting of 
mutually orthogonal vectors. 

A.4 Norms and normalization 

A norm is a function that measures the length of vectors. In a finite- 
dimensional vector space, we typically use the norm||M|| :=(u \ u) l ^ 2 . 
If we are working with a function space such as C[0, 1], we ordinar- 
ily use one of the LF norms, defined as 



1 x Up 

\u(x)\ p dx 



In the limit as p tends to infinity, we get what is known as the max- 
norm: 

||m|| := max \u{x)\. 

*6[0,1] 

Even more frequently used is theL 2 norm, which can also be written 
as ||w|| 2 = (u | w) 1/2 if we are using the standard inner product. 

A vector u with ||//|| = 1 is said to be normalized. If we have an 
orthogonal basis composed of vectors that are normalized in theL 2 
norm, the basis is called orthonormal. Stated concisely, a basis 
mi, U2, . . . is orthonormal if 

(Ui | Uj) = 5ij, 

where 5y is called the Kronecker delta and is defined to be 1 iff =j, 
and 0 otherwise. 

Example: The vectors e x = (1,0,0), e 2 = (0, 1,0), e 3 = 
(0, 0, 1) form an orthonormal basis for the inner product space 
R 3 endowed with the dot product of Equation (1). 
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